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Abstract. Nondeterministic weighted automata are finite automata with numerical weights 
on transitions. They define quantitative languages L that assign to each word w a real num- 
ber L{w). The value of an infinite word w is computed as the maximal value of all runs over w, 
and the value of a run as the maximum, limsup, liminf, limit average, or discounted sum of the 
transition weights. We introduce probabilistic weighted automata, in which the transitions are 
chosen in a randomized (rather than nondeterministic) fashion. Under almost-sure semantics 
(resp. positive semantics), the value of a word w is the largest real v such that the runs over w 
have value at least v with probability 1 (resp. positive probability). 

We study the classical questions of automata theory for probabilistic weighted automata: empti- 
ness and universality, expressiveness, and closure under various operations on languages. For 
quantitative languages, emptiness and universality are defined as whether the value of some 
(resp. every) word exceeds a given threshold. We prove some of these questions to be decid- 
able, and others undecidable. Regarding expressive power, we show that probabilities allow us 
to define a wide variety of new classes of quantitative languages, except for discounted-sum 
automata, where probabilistic choice is no more expressive than nondeterminism. Finally, we 
give an almost complete picture of the closure of various classes of probabilistic weighted au- 
tomata for the following pointwise operations on quantitative languages: max, min, sum, and 
numerical complement. 



1 Introduction 

In formal design, specifications describe the set of correct behaviours of a system. An implementa- 
tion satisfies a specification if all its behaviours are correct. If we view a behaviour as a word, then 
a specification is a language, i.e., a set of words. Languages can be specified using finite automata, 
for which a large number of results and techniques are known; see [19,23]. We call them boolean 
languages because a given behaviour is either good or bad according to the specification. Boolean 
languages are useful to specify functional requirements. 

In a generalization of this approach, we consider quantitative languages, where each word is 
assigned a real number The value of a word can be interpreted as the amount of some resource 
(e.g., memory or power) needed to produce it, or as a quality measurement for the corresponding 
behaviour [5, 6]. Therefore, quantitative languages are useful to specify non-functional requirements 
such as resource constraints, reliability properties, or levels of quality (such as quality of service). 

Quantitative languages can be defined using (nondeterministic) weighted automata, i.e., finite 
automata with numerical weights on transitions [12, 16]. In [7], we studied quantitative languages 
of infinite words and defined the value of an infinite word w as the maximal value of all runs of an 
automaton over w (if the automaton is nondeterministic, then there may be many runs over w). The 



(a) Low reliability but cheap. 



(b) High reliability but expensive. 



Fig. 1. Two specifications of a channel. 



value of a run r is a function of the infinite sequence of weights that appear along r. There are several 
natural functions to consider, such as Sup, LimSup, Liminf, limit average, and discounted sum of 
weights. For example, peak power consumption can be modeled as the maximum of a sequence of 
weights representing power usage; energy use, as a discounted sum; average response time, as a limit 
average [4,5]. 

In this paper, we consider probabilistic weighted automata as generators of quantitative lan- 
guages. In such automata, nondeterministic choice is replaced by probability distributions on suc- 
cessor states. The value of an infinite word w is defined to be the maximal value v such that the set of 
runs over w with value at least v has either positive probability (positive semantics), or probability 1 
(almost-sure semantics). This simple definition combines in a general model the natural quantitative 
extensions of logics and automata [13, 14,7], and the probabilistic models of automata for which 
boolean properties have been well studied [21, 3, 2]. Note that the probabilistic Biichi and coBuchi 
automata of [2] are a special case of probabilistic weighted automata with weights and 1 only 
(and the value of an infinite run computed as LimSup or Liminf, respectively). While quantitative 
objectives are standard in the branching-time context of stochastic games [22, 15, 17,5, 10, 18], we 
are not aware of any model combining probabilities and weights in the linear-time context of words 
and languages, though such a model is very natural for the specification of quantitative properties. 
Consider the specification of two types of communication channels given in Fig. 1. One has low cost 
(sending costs 1 unit) and low reliability (a failure occurs in 10% of the case and entails an increased 
cost for the operation), while the second is expensive (sending costs 5 units), but the reliability is 
high (though the cost of a failure is prohibitive). In the figure, we omit the self-loops with cost in 
state qo and q'q over ack, and in qi,q2,q[, q'2 over send. Natural questions can be formulated in this 
framework, such as whether the average-cost of every word w G {send, ack}'^ is really smaller in 
the low-cost channel, or to construct a probabilistic weighted automaton that assigns the minimum 
of the average-cost of the two types of channels. In this paper, we attempt a comprehensive study of 
such fundamental questions, about the expressive power, closure properties, and decision problems 
for probabilistic weighted automata. 

First, we compare the expressiveness of the various classes of probabilistic and nondeterministic 
weighted automata over infinite words. For LimSup, Liminf, and limit average, we show that a wide 
variety of new classes of quantitative languages can be defined using probabilities, which are not 
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expressible using nondeterminism. Our results rely on reachability properties of closed recurrent 
sets in Markov chains. For discounted sum, we show that probabilistic weighted automata under the 
positive semantics have the same expressive power as nondeterministic weighted automata, while 
under the almost-sure semantics, they have the same expressive power as weighted automata with 
universal branching, where the value of a word is the minimal (instead of maximal) value of all 
runs. The question of whether the positive semantics of weighted limit-average automata is more 
expressive than nondeterminism, remains open. 

Second, we give an almost complete picture of the closure of probabilistic weighted automata 
under the pointwise operations of maximum, minimum, and sum for quantitative languages. We also 
define the complement L'^ of a quantitative language L by L'^{w) = 1 — L{w) for all words 
Note that maximum and minimum are in fact the operation of least upper bound and greatest lower 
bound for the pointwise natural order on quantitative languages (where Li < L2 if and only if 
Li{w) < L2{w) for all words w). Therefore, they also provide natural generalization of the classical 
union and intersection operations of boolean languages. 

Note that closure under max trivially holds for the positive semantics, and closure under min 
for the almost-sure semantics. We also define the complement of a quantitative language L by 
L'^iw) = 1 — L(w) for all words w. Only LimSup-automata under positive semantics and Limlnf- 
automata under almost-sure semantics are closed under all four operations; these results extend cor- 
responding results for the boolean (i.e., non-quantitative) case [1]. To establish the closure properties 
of limit-average automata, we characterize the expected limit-average reward of Markov chains. Our 
characterization answers all closure questions except for the language sum in the case of positive 
semantics, which we leave open. Note that expressiveness results and closure properties are tightly 
connected. For instance, because they are closed under max, the Limlnf-automata with positive se- 
mantics can be reduced to Limlnf-automata with almost-sure semantics and to LimSup-automata 
with positive semantics; and because they are not closed under complement, the LimSup-automata 
with almost-sure semantics and Limlnf-automata with positive semantics have incomparable expres- 
sive powers. 

Third, we investigate the emptiness and universality problems for probabilistic weighted au- 
tomata, which ask to decide if some (resp. all) words have a value above a given threshold. Using 
our expressiveness results, as well as [1, 8], we establish some decidability and undecidability results 
for Sup, LimSup, and Liminf automata; in particular, emptiness and universaUty are undecidable 
for LimSup-automata with positive semantics and for Limlnf-automata with almost-sure semantics, 
while the question is open for the emptiness of Limlnf-automata with positive semantics and for 
the universality of LimSup-automata with almost-sure semantics. We also prove the decidability of 
emptiness for probabilistic discounted-sum automata with positive semantics, while the universality 
problem is as hard as for the nondeterministic discounted-sum automata, for which no decidability 
result is known. We leave open the case of limit average. 

2 Definitions 

A quantitative language over a finite alphabet 17 is a function L : ^ K. A boolean language (or a 
set of infinite words) is a special case where L{'w) G {0, 1} for all words w G Z"". Nondeterministic 
weighted automata define the value of a word as the maximal value of a run [7]. In this paper, we 
study probabilistic weighted automata as generator of quantitative languages. 

One can define L'^{w) = k — L{w) for any constant k without changing the results of this paper. 
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Value functions. We consider the following value functions Val : Q'^ ^ M to define quantitative 
languages. Given an infinite sequence v = vqVi ... of rational numbers, define 

- Sup(u) = sup{u„ \ n>0}; 

- LimSup(w) = limsup u„ = lim supjwi | i > n}; 



n — >oc 



n — 'oo 



- Limlnf(u) ~ liminf w„ = lim ini{vi \ i > n}; 

n — >oo n — >oo 

Tl-1 



2 " " 

- LimAvg(w) = liminf — \^ vf, 

4=0 

oc 

- For < A < 1, DiscA(i') = ■ Vi\ 



i=0 

Given a finite set S, a probabilistic distribution over 5 is a function / : S ~> [0,1] such that 
X]sgS /('^) ^ ^- denote by the set of all probabilistic distributions over S. 
Probabilistic weighted automata. A probabilistic weighted automaton is a tuple A = 
{Q,pi, 5,7) where: 

- Q is a finite set of states; 

- pi & T^iQ) is the initial distribution; 

- is a finite alphabet; 

- S : Q X S T^iQ) is a probabilistic transition function; 

- 7:QxZ'xQ^Qisa weight function. 

We can define a non-probabilistic automaton from A by ignoring the probability values, and 
saying that (? is initial if > 0, and (g, cr, q') is an edge of A if (t)((7') > 0. The automaton A 
is deterministic if pi{qi) = 1 for some qi G Q, and for all q G Q and <t G there exists q' ^ Q 
such that (5(g, (T){q') = 1. 

A run of A over a finite (resp. infinite) word u> = cri(T2 . . . is a finite (resp. infinite) sequence 
r = goO'iQiO'2 ... of states and letters such that (i) pi{qo) > 0, and (ii) S{qi, ai+i,qi+i) > for 
all < I < \w\. We denote by 7(r) = vqVi . . . the sequence of weights that occur in r where 
Vi = l[qi, (Ji+i,qi+i) for all < i < \w\. 

The probability of a finite run r = qvi(Tiqi(T2 ■ ■ ■ (Jkqk over a finite word w = ai . . .ct/c is 
P^(r) = pj[qQ).Y['^^iS{qi-i,ai){qi). For each w G S'^, the function P'^(-) defines a unique 
probability measure over Borel sets of (infinite) runs of A over w. 

Given a value function Val : Q'^ M, we say that the probabilistic Val -automaton A gener- 
ates the quantitative languages defined for all words w G by L^^(w) = supjjy | ¥^{{r G 
Run (w) I Val(7(r)) > 77}) = 1} under the almost-sure semantics, and L^^{'w) = supjjy 
P"^({r G Run"*(w) | Val(7(r)) > 77}) > 0} under the positive semantics. For non-probabilistic au- 
tomata, the value of a word is either the maximal value of the runs (i.e., L™^^(w) = sup{Val(7(r)) 
r G Run'^(w)} for all w G X!'^) and the automaton is then called nondeterministic, or the minimal 
value of the runs, and the automaton is then called universal. 

Note that Buchi and coBuchi automata ([2]) are special cases of respectively LimSup- and 
Limlnf-automata, where all weights are either or 1. 

Notations. The first letter in acronyms for classes of automata can be N(ondeterministic), 
D(eterministic), U(niversal), Pos for the language in the positive semantics, or As for the language 
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AsLimAvg PosLimAvg - NLimAvg PosLimInf NLimSup AsLimSup 
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Fig. 2. Reducibility relation. C is reducible to C' if C — » C'. Classes that are not connected by an arrow are in- 
comparable. Reducibility for the dashed arrow is open. The Disc-automata are incomparable with the automata 
in the figure. Their reducibility relations are given in Theorem 5. 



in the almost-sure semantics. We use the notations ° to denote the classes of automata whose deter- 
ministic version has the same expressiveness as their nondeterministic version. When the type of an 
automaton A is clear from the context, we often denote its language simply by La{-) or even A{-), 
instead of L^, L™""^, etc. 

Reducibility. A class C of weighted automata is reducible to a class C of weighted automata if for 
every A ^ C there exists A' G C' such that La = La', i.e. La{w) = La'{w) for all words w. 
Reducibility relationships for (non)deterministic weighted automata are given in [7]. 

Composition. Given two quantitative languages L,L' : K, we denote by ma.x{L, L') 

(resp. mm{L,L') and L + L') the quantitative language that assigns max{L{w) , L' (w)} (resp. 
mm{L{w), L' (w)} and L{w) + L'{w)) to each word w G S'^. The language 1 — L is called the 
complement of L. The max, min and complement operators for quantitative languages generalize 
respectively the union, intersection and complement operator for boolean languages. The closure 
properties of (non)deterministic weighted automata are given in [8]. 

Remark. We sometimes use automata with weight functions 7 : Q ^ Q that assign a weight 
to states instead of transitions. This is a convenient notation for weighted automata in which from 
each state, all outgoing transitions have the same weight. In pictorial descriptions of probabilistic 
weighted automata, the transitions are labeled with probabilities, and states with weights. 



3 Expressive Power of Probabilistic Weighted Automata 

We complete the picture given in [7] about reducibility for nondeterministic weighted automata, by 
adding the relations with probabilistic automata. The results for Liminf, LimSup, and LimAvg are 
summarized in Fig. 2s, and for Sup- and Disc-automata in Theorems 1 and 5. 
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3.1 Probabilistic Sup-automata 

Like for probabilistic automata over finite words, the quantitative languages definable by probabilis- 
tic and (non)deterministic Sup-automata coincide. 

Theorem 1. PosSuP and AsSuP are reducible to DSUP. 

Proof. It is easy to see that PosSuP-automata define the same language when interpreted as NSUP- 
automata, and the same holds for AsSuP and USUP. The result then follows from [7, Theorem 9]. 



3.2 Probabilistic LimAvg-automata 

Many of our results would consider Marfcov chains and closed recurrent states in Markov chains. A 
Markov chain M = {S, E, 6) consists of a finite set S of states, a set E of edges, and a probabilistic 
transition function S : S ^ T^iS)- For all s, t ^ S, there is an edge (s, t) E E iff S{s){t) > 0. A 
closed recurrent set C of states in M is a bottom strongly connected set of states in the graph {S, E). 
We will use the following two key properties of closed recurrent states. 

1. Property 1. Given a Markov chain AI, and a start state s, with probability 1, the set of closed 
recurrent states is reached from s in finite time. Hence for any e > 0, there exists fcg such that for 
all k > ko, for all starting state s, the set of closed recurrent states are reached with probability 
at least 1 — e in fc steps. 

2. Property 2. If a closed recurrent set C is reached, and the limit of the expectation of the average 
weights of C is a, then for all e > 0, there exists a ko such that for all k > ko the expectation of 
the average weights for k steps is at least a — e. 

The above properties are the basic properties of finite state Markov chains and closed recurrent 
states [20]. 

Lemma 1. Let A be a probabilistic weighted automata with alphabet U = {a, 6}. Consider the 
Markov chain arising of A on input (we refer to this as the b-Markov chain) and we use similar 
notation for the a-Markov chain. The following assertions hold: 

1. If for all closed recurrent sets C in the b-Markov chain, the (expected) limit-average value (in 
probabilistic sense) is at least 1, then there exists j such that for all closed recurrent sets arising 
of A on input {V ■ a)^ the expected limit-average reward is positive. 

2. If for all closed recurrent sets C in the b-Markov chain, the ( expected) limit-average value ( in 
probabilistic sense) is at most 0, then there exists j such that for all closed recurrent sets arising 
of A on input ■ a)^ the expected limit-average reward is strictly less than 1. 

3. If for all closed recurrent sets C in the b-Markov chain, the ( expected) limit-average value ( in 
probabilistic sense) is at most 0, and if for all closed recurrent sets C in the a-Markov chain, the 
(expected) limit-average value (in probabilistic sense) is at most 0, then there exists j such that 
for all closed recurrent sets arising of A on input {V ■ a^)^ the expected limit-average reward 
is strictly less than 1/2. 

Proof. We present the proof in three parts. 
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Fig. 3. A PosLimAvg for Lemma 2. 

1. Let p be the maximum absolute value of the weights of A. From any state s G A, there is a path 
of length at most ri to a closed recurrent set C in the 6-Markov chain, where n is the number of 
states of A. Hence if we choose j > n, then any closed recurrent set in the Markov chain arising 
on the input (6-' • a)" contains closed recurrent sets of the 6-Markov chain. For e > 0, there exists 
fcf such that from any state s G A, for all k > k^,on input 6*^ from s, the closed recurrent sets of 
the 6-Markov chain is reached with probability at least 1 — e (by property 1 for Markov chains). 
If all closed recurrent sets in the 6-Markov chain have expected limit-average value at least 1, 
then (by property 2 for Markov chains) for all e > 0, there exists such that for all I > l^, from 
all states s of a closed recurrent set on the input 6' the expected average of the weights is at least 
1 — e, (i.e., expected sum of the weights isl ~ I ■ e). Consider < e < min{l/4, 1/(20 ■ f3)}, we 
choose j — k + 1, where k = fc^ > and I > maxjZj , k}. Observe that by our choice j + 1 < 21. 
Consider a closed recurrent set in the Markov chain on (6^ • a)" and we obtain a lower bound 
on the expected average reward as follows: with probability 1 — e the closed recurrent set of the 
6-Markov chain is reached within k steps, and then in the next I steps at the expected sum of the 
weights is at least I — I ■ e, and since the worst case weight is —(3 we obtain the following bound 
on the expected sum of the rewards 

/ / 2/ 

(l_,).(;_r,)_,./3.(^- + l)>___^_ 

Hence the expected average reward is at least 1/5 and hence positive. 

2. The proof is similar to the previous result. 

3. The proof is also similar to the first result. The only difference is that we use a long enough 
sequence of 6^ such that with high probability a closed recurrent set in the 6-Markov chain is 
reached and then stay long enough in the closed recurrent set to approach the expected sum of 
rewards to 0, and then present a long enough sequence of such that with high probability a 
closed recurrent set in the a-Markov chain is reached and then stay long enough in the closed 
recurrent set to approach the expected sum of rewards to 0. The calculation is similar to the first 
part of the proof. 

Thus we obtain the desired result. ■ 

We consider the alphabet S consisting of letters a and 6, i.e., S = {a, 6}. We define the language 
Lp of finitely many as, i.e., for an infinite word wifw consists of infinitely many a's, then Lp{w) = 
0, otherwise Lp{w) = 1. We also consider the language Lj of words with infinitely many a's (it is 
the complement of Lp). 
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Lemma 2. Consider the language Lp of finitely many a's. The following assertions hold. 

1. The language can be expressed as a NLimAvg. 

2. The language can be expressed as a PosLimAvg. 

3. The language cannot be expressed as AsLimAvg. 

Proof. We present the three parts of the proof. 

1. The result follows from the results of [7, Theorem 12] where the explicit construction of a 
NLimAvg to express Lj;- is presented. 

2. A PosLimAvg automaton A to express is as follows (see Fig. 3): 

(a) States and weight function. The set of states of the automaton is {go, 9i, sink}, with qq as 
the starting state. The weight function 7 is as follows: 'y{qo) = j{sink) = and 7(51) = 1. 

(b) Transition function. The probabilistic transition function is as follows: 

(i) from qo, given a or b, the next states are qo,qi, each with probability 1/2; 

(ii) from qi given b the next state is qi with probability 1, and from qi given a the 
next state is sink with probability 1 ; and 

(iii) from sink state the next state is sink with probability 1 on both a and b. (it is 
an absorbing state). 

Given the automaton A consider any word w with infinitely many a's then, the automata reaches 
sink state in finite time with probability 1, and hence A{w) = 0. For a word w with finitely many 
a's, let k be the last position that an a appears. Then with probability 1/2*^, after k steps, the 
automaton only visits the state qi and hence A{w) = 1. Hence there is a PosLimAvg for Lp. 

3. We show that Lp cannot be expressed as an AsLimAvg. Consider an AsLimAvg automaton 
A. Consider the Markov chain that arises from A if the input is only b (i.e., on b'^), we refer to it 
as the 6-Markov chain. If there is a closed recurrent set C that can be reached from the starting 
state (reached by any sequence of a and 6's), then the limit-average reward (in probabilistic 
sense) in C must be at least 1 (otherwise, if there is a closed recurrent set C with limit-average 
reward less than 1, we can construct a finite word w that with positive probabiUty will reach C, 
and then follow w by 6" and we will have A{w ■ b'^) < 1). Hence any closed recurrent set on 
the 6-Markov chain has limit-average reward at least 1 and by Lemma 1 there exists j such that 
the A{{V ■ a)'^) > 0. Hence it follows that A cannot express Lp. 

Hence the result follows. ■ 

Lemma 3. Consider the language Lj of infinitely many a's. The following assertions hold. 

L The language cannot be expressed as an NLimAvg. 

2. The language cannot be expressed as a PosLimAvg. 

3. The language can be expressed as AsLimAvg. 

Proof. We present the three parts of the proof. 
1. It was shown in the proof of [7, Theorem 13] that NLimAvg cannot express Lj. 
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Fig. 4. An AsLimAvg for Lemma 3. 




Fig. 5. A probabilistic weighted automaton (PosLimAvg, PosLimSup, or PosLlMiNF) for Lemma 4. 

2. We show that Lj is not expressible by a PosLimAvg. Consider a PosLimAvg A and consider 
the 6-Markov chain arising from A under the input b'^. All closed recurrent sets C reachable 
from the starting state must have the limit-average value at most (otherwise we can construct 
an word w with finitely many a's such that A{w) > 0). Since all closed recurrent set in the 
6-Markov chain has limit-average reward that is 0, using Lemma 1 we can construct a word 
w = {V ■ a)'^, for a large enough j, such that A{w) < 1. Hence the result follows. 

3. We now show that Lj is expressible as an AsLimAvg. The automaton A is as follows (see 
Fig. 4): 

(a) States and weight function. The set of states are {go, sink} with go as the starting state. The 
weight function is as follows: j{qo) ~ and j{sink) ~ 1. 

(b) Transition function. The probabilistic transition function is as follows: 

(i) from qo given b the next state is qq with probability 1 ; 

(ii) at qo given a the next states are go and sink each with probability 1/2; (iii) the 
sink state is an absorbing state. 

Consider a word w with infinitely many a's, then the probability of reaching the sink state is 1, 
and hence A{w) = 1. Consider a word w with finitely many a's, and let k be the number of a's, 
and then with probability 1/2'"' the automaton always stay in qo, and hence A{w) = 0. 

Hence the result follows. ■ 



Lemma 4. There exists a language that can be expressed by PosLimAvg, PosLimSup and 
PosLlMiNF, but not by NLimAvg, NLimSup or NLlMlNF. 

Proof. Consider an automaton A as follows (see Fig. 5): 
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1. States and weight function. The set of states are {go, 91 7 sink} with as the starting state. The 
weight function is as follows: ^{qo) = 7((?i) = 1 and ^{sink) = 0. 

2. Transition function. The probabilistic transition is as follows: 

(i) from go if the input letter is a, then the next states are qo and qi with probability 1/2; 

(ii) from go if the input letter is b, then the next state is sink with probability 1 ; 

(iii) from gi, if the input letter is b, then the next state is go with probability 1; 

(iv) from gi, if the input letter is a, then the next state is gi with probability 1; and 

(v) the state sink is an absorbing state. 

If we consider the automaton A, and interpret it as a PosLimAvg, PosLimSup, or PosLlMiNF, 
then it accepts the following language: 

00 

L, = {a'''ba''^ba'''b... \ ki,k2,--- e N>i • - ^) > 0} U {a U b)* ■ a'^ ; 

i=l 

i.e., A{w) = 1 if e and A{w) = if w ^ L^: the above claim follows easily from the argument 
following Lemma 5 of [2]. We now show that Lz cannot be expressed as NLimAvg, NLimSup or 
NLimInf. Consider a non-deterministic automaton A. Suppose there is a cycle C in A such that 
average of the rewards in C is positive, and C is formed by a word that contains a b. If no such 
cycle exists, then clearly A cannot express as there exists word for which (w) = 1 such that 
w contains infinitely many 6's. Consider a cycle C such that average of the rewards is positive, and 
let the cycle be formed by a finite word wc — aoai ■ ■ - o-n and there must exist at least one index 
< i < n such that — b. Hence the word can be expressed as wc = a^^ ba^^ b . . . a^'' b, and hence 
there exists a finite word wr (that reaches the cycle) such that A{wu ■ Wq) > 0. This contradicts 
that A is an automaton to express as Lz{wr ■ Wq) = 0. Simply exchanging the average reward 
of the cycle by the maximum reward (resp. minimum reward) shows that is not expressible by a 
NLimSup (resp. NLimInf). ■ 

The next theorem summarizes the results for limit-average automata obtained in this section. 

Theorem 2. AsLimAvg is incomparable in expressive power with PosLimAvg and NLimAvg, 
and NLimAvg cannot express all languages expressible by PosLimAvg. 

Open question. Whether NLimAvg is reducible to PosLimAvg or NLimAvg is incomparable to 
PosLimAvg (i.e., there is a language expressible by NLimAvg but not by a PosLimAvg) remains 
open. 

3.3 Probabilistic Limlnf-automata 

Lemma 5. NLimInf is reducible to both AsLlMiNF and PosLlMlNF. 

Proof. It was shown in [7] that NLlMiNF is reducible to DLlMlNF. Since DLlMiNF are special 
cases of AsLimInf and PosLimInf the result follows. ■ 

Lemma 6. The language Lj is expressible by an AsLlMiNF, but cannot be expressed as a NLlMiNF 
or a PosLimInf. 
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Proof. It was shown in [7] that the language Lj is not expressible by NLlMlNF. If we consider 
the automaton A of Lemma 3 and interpret it as an AsLlMlNF, then the automaton A expresses the 
language Lj. The proof of the fact that PosLlMlNF cannot express Li is similar to the the proof 
of Lemma 3 (part(2)) and instead of the average reward of the closed recurrent set C, we need to 
consider the minimum reward of the closed recurrent set C. ■ 



Lemma 7. PosLimInf is reducible to AsLlMlNF. 



Proof. Let A be a PosLlMlNF and we construct a AsLlMlNF B such that B is equivalent to A. Let 
V be the set of weights that appear in A and let vi be the least value in V . For each weight v ^V, 
consider the PosCW A" that is obtained from A by considering all states with weight at least v as 
accepting states. It follows from the results of [1] that PosCW is reducible to AsCW (it was shown 
in [1] that AsBW is reducible to PosBW and it follows easily that dually PosCW is reducible to 
AsCW). Let D" be an AsCW that is equivalent to A". We construct a PosLlMlNF B^ from 
by assigning weights v to the accepting states of and the minimum weight vi to all other states. 
Consider a word w, and we consider the following cases. 

1. If A{w) = V, then for all v' e V such that w' < v we have D""' {w) = 1, (i.e., the PosCW A"' 
and the AsCW D"' accepts w). 

2. For V eV,\f D"{w) = 1, then A{w) > v 

It follows from above that A = max„gy B". We will show later that AsLlMiNF is closed under 
max (Lemma 18) and hence we can construct an AsLlMiNF B such that B = max„gy B^. Thus 
the result follows. ■ 

Theorem 3. We have the following strict inclusion 

NLlMiNF C PosLlMlNF C AsLIMINF 



Proof. The fact that NLlMiNF is reducible to PosLlMlNF follows from Lemma 5, and the fact 
the PosLlMlNF is not reducible to NLlMiNF follows from Lemma 4. The fact that PosLlMlNF 
is reducible to AsLlMiNF follows from Lemma 7 and the fact that AsLlMiNF is not reducible to 
PosLlMlNF follows from Lemma 6. ■ 



3.4 Probabilistic LimSup-automata 

Lemma 8. NLimSup and PosLimSup are not reducible to AsLimSup. 

Proof. The language Lp of finitely many a's can be expressed as a non-deterministic Biichi au- 
tomata, and hence as a NLimSup. We will show that NLimSup is reducible to PosLimSup. It 
follows that Li? is expressible as NLimSup and PosLimSup. The proof of the fact that AsLimSup 
cannot express Lp is similar to the the proof of Lemma 2 (part(3)) and instead of the average reward 
of the closed recurrent set C, we need to consider the maximum reward of the closed recurrent set 

c. m 
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Deterministic in limit NLimSup. Consider an automaton A that is a NLimSup. Let vi < V2 < 
... < Vfc be the weights that appear in A. We call the automaton A deterministic in the limit if for 
all states s with weight greater than vi, all states t reachable from s are deterministic. 

Lemma 9. For every NLimSup A, there exists a NLimSup B that is deterministic in the limit and 
equivalent to B. 

Proof. From the results of [11] it follows that a NBW A can be reduced to an equivalent NBW B 
such that B is deterministic in the limit. Let A be a NLimSup, and let V be the set of weights that 
appear in A. and let V = {ui, . . . ,Vk} with vi < V2 < ■ ■ ■ < Vk- For each v ^ V, consider the 
NBW Ay whose (boolean) language is the set of words w such that La{w) > v, by declaring to 
be accepting the states with weight at least v. Let By be the deterministic in the limit NBW that is 
equivalent to Ay . The automaton B that is deterministic in the limit and is equivalent to A is obtained 
as the automaton that by initial non-determinism chooses between the By's, for v <E V. ■ 

Lemma 10. NLimSup is reducible to PosLimSup. 

Proof. Given a NLimSup A, consider the NLimSup B that is deterministic in the limit and equiv- 
alent to B. By assigning equal probabilities to all out-going transitions from a state we obtain a 
PosLimSup C that is equivalent to B (and hence A). The result follows. ■ 

Lemma 11. AsLimSup is reducible to PosLimSup. 

Proof. Consider a AsLimSup A and let the weights of A be ui < V2 ■ ■ ■ < vi. For 1 < i < I 
consider the AsB W obtained from A with the set of state with reward at least Vi as the Biichi states. 
It follows from the results of [1] that AsBW is reducible to PosBW. Let B^ be the PosBW that 
is equivalent to Ai. Let Ci be the automaton such that all Biichi states of Bi is assigned weight Vi 
and all other states are assigned vi . Consider the automata C that goes with equal probability to the 
starting states of Ci, for 1 < i < /, and we interpret C as a PosLimSup. Consider a word w, and 
let A{w) — Vj for some I < j < I, i.e., given w, the set of states with reward at least Vj is visited 
infinitely often with probability 1 in A. Hence the PosBW Bi accepts w with positive probability, 
and since C chooses Ci with positive probability, it follows that given w, in C the weight Vj is 
visited infinitely often with positive probability, i.e., C{w) > Vj. Moreover, given w, for all Vk > vi, 
the set of states with weight at least Vk is visited infinitely often with probability in A. Hence for 
all k > j, the automata Bk accepts w with probability 0. Thus C{w) < Vk for all Vk > Vj. Hence 
C{w) = A{w) and thus AsLimSup is reducible to PosLimSup. ■ 

Lemma 12. AsLimSup is not reducible to NLimSup. 

Proof. It follows from [1] that for < A < 1 the following language L\ can be expressed by a 
AsBW and hence by AsLimSup: 

oo 

La = {a'''ba''^ba'''>b... \ fci, fe, • • ■ e N>i. ]^(1 - A*^') > 0}. 

1=1 
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It follows from argument similar to Lemma 4 that there exists < A < 1 such that Lx cannot be 
expressed by a NLimSup. Hence the result follows. ■ 



Theorem 4. AsLimSup and NLimSup are incomparable in expressive power, a«ti PosLimSup is 
more expressive than AsLimSup and NLimSup. 

Lemma 13. PosCW is reducible to PosBW. 

Proof. Let A = {Q,qj,S,5, C) be a PosCW with the set C C Q of accepting states. We construct 
a PosBW 'A as follows: 

1. The set of states is Q U Q where Q = {g | g G Q} is a copy of the states in Q; 

2. qi is the initial state; 

3. The transition function is as follows, for all a G S:_ 

(a) for all states g, q' G Q, we have 5{q^ cr, q') = 5{q, a, q') ^ \ ■ S{q, cr, q'), i.e., the state q' 
and its copy g' are reached with half of the original transition probability; 

(b) the states q € Q such that g ^ C are absorbing states (i.e., S{q, cr, q) ~ 1); 

(c) for all states q G C and q' G Q, we have S{q, cr, q') ~ S{q, cr, q'), i.e., the transition function 
in the copy automaton follows that of_A for states that are copy of the accepting states. 

4. The set of accepting states is C = {g G Q | g G C}. 

We now show that the language of the PosCW A and the language of PosBW A coincides. Consider 
a word w such that A{w) ^ 1. Let a be the probability that given the word w eventually always states 
in C are visited in A, and since A{w) = 1 we have o; > 0. In other words, as limit k tends to cxd, 
the probability that after k steps only states in C are visited is a. Hence there exists ko such that 
the probability that after ko steps only states in C are visited is at least ^. In the automaton A, 
the probability to reach states of Q after fco steps has probability p = 1 — > 0. Hence with 
positive probability (at least p • §) the automaton visits infinitely often the states of C, and hence 
A{w) = 1. Observe that since every state in Q \ C is absorbing and non-accepting), it follows that if 
we consider an accepting run A, then the run must eventually always visits states in C (i.e., the copy 
of the accepting states C). Hence it follows that for a given word w, if A{w) = 1, then with positive 
probabiUty eventually always states in C are visited in A. Thus A{w) ~ 1, and the result follows. 



Lemma 14. PosLimInf is reducible to PosLimSup, and AsLimSup is reducible to AsLlMlNF. 

Proof. We present the proof that PosLlMiNF is reducible to PosLimSup, the other proof being 
similar. Let A be a PosLlMiNF, and let V be the set of weights that appear in A. For each v G V, 
it is easy to construct a PosCW Ay whose (boolean) language is the set of words w such that 
La{w) > V, by declaring to be accepting the states with weight at least v. We then construct for 
each V G V a PosBW Ay that accepts the language of A^ (such a PosBW can be constructed by 
Lemma 13). Finally, assuming that V = {vi, . . . , w„} with vi < V2 < ■ ■ ■ < we construct the 
PosLimSup Bi for i = 1,2, ... ,n where Bi is obtained from Ay. by assigning weight Vi to each 
accepting states, and vi to all the other states. The PosLimSup that expresses the language of A is 
maxj-i^2....n Bi and since PosLimSup is closed under max (see Lemma 16), the result follows. 
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Lemma 15. AsLimInf and PosLimSup are reducible to each other; AsLimSup and PosLlMiNF 
have incomparable expressive power 

Proof. This result is an easy consequence of the fact that an automaton interpreted as AsLlMiNF 
defines the complement of the language of the same automaton interpreted as PosLimSup (and 
similarly for AsLimSup and PosLimInf), and from the fact that AsLiMiNF and PosLimSup are 
closed under complement, while AsLimSup and PosLimInf are not (see Lemma 21 and 22). 



3.5 Probabilistic Disc-automata 

For probabilistic discounted-sum automata, the following result establishes equivalence of the non- 
deterministic and the positive semantics, and the equivalence of the universal and the almost-sure 
semantics. 

Theorem 5. The following assertions hold: (a) NDiSC and PosDiSC are reducible to each other; 
(b) UDiSC and AsDiSC are reducible to each other 

Proof, (a) We first prove that NDiSC is reducible to PosDiSC. Let A = {Q, pj, S, Sa, 7) be a 
NDiSC, and let Umin, i^max be its minimal and maximal weights respectively. Consider the PosDiSC 
B = {Q, Pj, S, Sb,j) where Ssiq, cr) is the uniform distribution over the set of states q' such that 
{q, fT, {q'}) G Sa- Let r ~ qo(Jiqi(y2 ■ ■ ■ be a run of A (over w = (Ti(T2 . • . ) with value rj. For 
all e > 0, we show that ¥^{{r e Run-^(w) | Val(7(r)) > ?7 - e}) > 0}. Let n e N such that 
jtr\ ' (''^max ~ ''^min) < ^ud let r„ = (7oO'i'?if2 • ■ • <^nqn- The discouutcd sum of the weights in r„ 
is at least 77 — • (wmax)- The probability of the set of runs over w that are continuations of r„ 
is positive, and the value of all these runs is at least 77 — ' (*^max ^ timin), and therefore at least 
T] — e. This shows that Lb{w) > 77, and thus Lb{w) > La{w). Note that Lb{w) < La{w) since 
there is no run in A (nor in B) over w with value greater than La{w). Hence Lb = La- 

Now, we prove that PosDiSC is reducible to NDlSC. Given a PosDiSC B = {Q, pj, S, Sb, 7), 
we construct a NDiSC A = {Q, pj , S,Sa,j) v^heie {q, a, {q'}) & Sa if and only if (5b((7, (T)(g') > 
0, for all q,q' £ Q, a S. By analogous arguments as in the first part of the proof, it is easy to see 
that Lb ~ La- 

(b) It is easy to see that the complement of the quantitative language defined by a UDiSC (resp. 
AsDiSC) can be defined by a NDiSC (resp. PosDiSC). Then, the result follows from Part a) (essen- 
tially, given a UDiSC, we obtain easily an NDiSC for the complement, then an equivalent PosDiSC, 
and finally a AsDiSC for the complement of the complement, i.e., the original quantitative lan- 
guage). ■ 

Note that a by-product of this proof is that the language of a PosDiSC does not depend on the 
precise values of the probabilities, but only on whether they are positive or not. 

4 Closure Properties of Probabilistic Weighted Automata 

We consider the closure properties of the probabilistic weighted automata under the operations max, 
min, complement, and sum. The results are presented in Table 1 . 
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The universality problem for NDiSC can be reduced to (1). It is 
not known whether this problem is decidable. 



Table 1. Closure properties and decidability of the emptiness and universality problems. 



4.1 Closure under max and min 

Lemma 16 (Closure by initial non-determinism). PosLimSup, PosLimInf and PosLimAvg is 
closed under max; and AsLimSup, AsLimInf and AsLimAvg is closed under min. 

Proof. Given two automata Ai and A2 consider the automata A obtained by initial non-deterministic 
choice of Ai and A2. Formally, let qi and (72 be the initial states of Ai and A2, respectively, then 
in A we add an initial state go and the transition from is as follows: for a & E, consider the 
set Qcr = {g e Qi U Q2 | (5i(gi, cr)(g) > or (52((j2, o')(9) > 0}. From go, for input letter cr, 
the successors are from Q„ each with probability l/|Qcr|- If Ai and A2 are PosLimSup (resp. 
PosLimInf, PosLimAvg), then ^ is a PosLimSup (resp. PosLimInf, PosLimAvg) such that 
A = max{^i, ^2}- Similarly, if Ai and A2 are AsLimSup (resp. AsLimInf, AsLimAvg), then 
A is a AsLimSup (resp. AsLimInf, AsLimAvg) such that A = min{Ai, A2}. ■ 

Lemma 17 (Closure by synchronized product). AsLimSup is closed under max and PosLim- 
Inf is closed under min. 

Proof. We present the proof that AsLimSup is closed under max. Let Ai and A2 be two probabilis- 
tic weighted automata with weight function 71 and 72, respectively. Let A be the usual synchronized 
product of Ai and A2 with weight function 7 such that 7((si, S2)) — max{7i(si), 72(52)}- Given 
a path TT = ((.sj, Sq), {s\, ), . . .) in A we denote by tt f 1 the path in Ai that is the projec- 
tion of the first component of tt and we use similar notation for tt |" 2. Consider a word w, let 
max{^i(t(;), j42(w)} = v. We consider the following two cases to show that A{w) = v. 

1. W.l.o.g. let the maximum be achieved by Ai, i.e., Ai{w) = v. Let be the set of states Si 
in Ai such that weight of Si is at least v. Since Ai{w) = v, given the word w, in Ai the event 
Buchi(i3") holds with probability 1. Consider the following set of paths in A 

n" = {tt I (tt r 1) e Buchi(Bn}. 
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Since given w, the event Buchi(i3]') holds with probability 1 in Ai, it follows that given w, the 
event U'" holds with probability 1 in A. The 7 function ensures that every path tt G U" visits 
weights of value at least v infinitely often. Hence A{w) > v. 
2. Consider a weight value v' > v. Let C" be the set of states s,; in Ai such that the weight of 
Si is less than v'. Given the word w, since Ai{w) < v', it follows that probability of the event 
coBuchi(Cj") in Ai, given the word w, is positive. Hence given the word w, the probability of 
the event coBuchi(C" x C2 )) is positive in A. It follows that A{w) < v'. 

The result follows. If Ai and A2 are PosLlMiNF, and in A we assign weights such that every state 
in A has the minimum weight of its component states, and we consider A as a PosLlMiNF, then 
A = min{Ai, A2}. The proof is similar to the result for AsLimSup. ■ 

Lemma 18. PosLimSup is closed under min and AsLlMiNF is closed under max. 

Proof. Let Ai and A2 be two PosLimSup. We construct a PosLimSup A such that A = 
min{^i , ^2}- Let Vi be the set of weights that appear in Ai (for i ~ 1^2), and let V — V1UV2 and 
let vi be the least value in V. For each weight v E Vi UV2 — {vi, . . . , Vk}, consider the PosBW 
that is obtained from Ai by considering all states with weight at least v as accepting states. Since 
PosBW is closed under intersection(by the results of [2]), we can construct a PosBW AI2 that is 
the intersection of A^ and A!^, i.e. A^^ = A^" n A^. We construct a PosLimSup B'l^ from AI2 by 
assigning weights v to the accepting states of AI2 and the minimum weight vi to all other states. 
Consider a word w, and we consider the following cases. 

1. If rmn{Ai{w), A2{w)} ^ v, then for all v' E V such that w' < u we have Ai2iw) = 1, (i.e., 
the PosBW A12 accepts w). 

2. If Al2iw) = 1, then Ai{w) > v and A2{w) > v, i.e., min{^i('u;), A2(w)} > v. 

It follows from above that min{y4i, A2} = maxygy Bi2- Since PosLimSup is closed under max 
(by initial non-determinism), it follows that PosLimSup is closed under min. The proof of closure 
of AsLimInf under max is similar. ■ 

The closure properties of LimAvg-automata in the positive semantics rely on the following lemma. 

Lemma 19. Consider the alphabet S = {a, 5}, and consider the languages La and Li, that assigns 
the long-run average number of a's andb's, respectively. Then the following assertions hold. 

1. There is no PosLlMAVG/or the language L,„ = min{L(j, Lb\. 

2. There is no PosLimAvg /or the language L* — 1 ~ max{La, Lb\. 

Proof. To obtain a contradiction, assume that there exists a PosLimAvg A (for either L,„ or L*). 
We first claim that if we consider the a-Markov or the 5-Markov chain of A, then there must be either 
an a-closed recurrent set or a 5-closed recurrent set C that is reachable in A such that the expected 
sum of the weights in C is positive. Otherwise, if for all a-closed recurrent sets and 6-closed recurrent 
sets we have that the expected sum of the weights is zero or negative, then we fool the automaton 
as follows. By Lemma 1, it follows that there exists a j such that A{{a^ ■ V)'^) < 1/2, however, 
Lm{w) = L*{w) = i, i.e., we have a contradiction. W.l.o.g., we assume that there is an a-closed 
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recurrent set C such that expected sum of weights of C is positive. Then we present the following 
word w: a finite word wc to reach the cycle C, followed by a'^; the answer of the automaton is 
positive, i.e., La{w) > 0, while Lm{w) = L*{w) = 0. Hence the result follows. ■ 



Lemma 20. PosLimAvg is not closed under min and AsLimAvg is not closed under max. 

Proof. The result for PosLimAvg follows from Lemma 19. We now show that AsLimAvg is not 
closed under max. Consider the alphabet S = {a, 6} and the quantitative languages La and Lf, that 
assign the value of long-run average number of a's and 6's, respectively. There exists DLimAvg (and 
hence AsLimAvg) for La and Lf,. We show that L„i = max(La, Lt) cannot be expressed by an 
AsLimAvg. By contradiction, assume that A is an AsLimAvg with set of states Q that defines 
Consider any a-closed recurrent C in A. The expected limit-average of the weights of the recurrent 
set must be 1, as if we consider the word w* = wc ■ a!^ where wc is a finite word to reach C, 
the value of w* in Lm is 1. Hence, the limit-average of the weights of all the reachable a-closed 
recurrent set C in A is L 

Given e > 0, there exists such that the following properties hold: 

1 . from any state of A, given the word a^' with probability 1 — e an a-closed recurrent set is reached 
(by property 1 for Markov chains); 

2. once an a-closed recurrent set is reached, given the word a^' , (as a consequence of property 2 
for Markov chains) we can show that the following properties hold: (a) the expected average of 
the weights is at least — e), and (b) the probability distribution of the states is with e of the 
probability distribution of the states for the word a^'^' (this holds as the probability distribution 
of states on words converges to the probabiUty distribution of states on the word a'^). 

Let > 1 be a number that is greater than the absolute maximum value of weights in A. We chose 
e > such that e < -^rp- Let j = 2 • je (such that satisfies the properties above). Consider 
the word (a^ • b^^)'^ and the answer by A must be |, as L„-i{{a^ ■ b^^)'^) = |. Consider the word 
w = (a^^ • b^^)'^ and consider a closed recurrent set in the Markov chain obtain from A on w. We 
obtain the following lower bound on the expected limit-average of the weights: (a) with probability 
at least 1 — e, after j/2 steps, a-closed recurrent sets are reached; (b) the expected average of the 
weights for the segment between a-* and a^^ is at least j • (1 — e); and (c) the difference in probability 
distribution of the states after and a^-' is at most e. Since the limit-average of the weights of 
(a^ • b^^)'^ is |, the lower bound on the limit-average of the weights is as follows 

(l-3-e).(^i±iJi^)-3.e-/3=(l-e)(f-f)-3.6./3 

>l-e-3-e-/3 

> 4 _ J_ 

— 5 10 

- 10 5- 



It follows that A{{a^^ ■ 6^^)") > |. This contradicts that A expresses L„ 



17 



4.2 Closure under complement 



Lemma 21. PosLimSup and AsLimInf are closed under complement. 

Proof. We first present the proof for PosLimSup. Let A be a PosLimSup, and let V be the 
set of weights that appear in A. For each v & V, \i is easy to construct a PosBW whose 
(boolean) language is the set of words w such that > v, by declaring to be accepting the 

states with weight at least v. We then construct for each u G V a PosB W Ay (with accepting states) 
that accepts the (boolean) complement of the language accepted by Ay (such a PosBW can be 
constructed since PosBW is closed under complementation by the results of [1]). Finally, assuming 
that V = {di, . . . , Vn] with wi < ■i;2 < ■ ■ ■ < Vn, we construct the PosLimSup Bi for i = 2, . . . , n 
where Bi is obtained from Ay. by assigning weight —vt-i to each accepting states, and — to 
all the other states. The complement of La is then maxjLsa, • ■ • , Lb„} which is accepted by a 
PosLimSup (since PosLimSup is closed under max). The result for AsLimInf is similar and it 
uses the closure of AsCW under complementation which can be easily proved from the closure 
under complementation of P O s B W . ■ 



Lemma 22. AsLimSup and PosLimInf are not closed under complement. 

Proof. It follows from Lemma 8 that the language Lp finitely many a's is not expressible by an 
AsLimSup, whereas the complement Lj of infinitely many a's is expressible as a DBW and hence 
as a AsLimSup. It follows from Lemma 6 that language Lj is not expressible as an PosLimInf, 
whereas its complement Lj? is expressible by a DCW and hence a PosLimInf. ■ 

Lemma 23. PosLimAvg and AsLimAvg are not closed under complement. 

Proof. The fact that PosLimAvg is not closed under complement follows from Lemma 19. We now 
show that AsLimAvg is not closed under complement. Consider the DLimAvg A over alphabet 
S = {a, h] that consists of a single self-loop state with weight 1 for a and for h. Notice that 
A{w.a^) ~ 1 and A{w.b'^) = for all w e E* . To obtain a contradiction, assume that there exists a 
AsLimAvg B such that B ^ 1 — A. For all finite words w G S*, let B{w) be the expected average 
weight of the finite run of B over w. Fix < e < i. For all finite words w, there exists a number 
riiy such that the average number of a's in w.b^'^ is at most e, and there exists a number rriyj such 
that B{w.a^'^^' ) < e (since B{w.a'^) = 0). Hence, we can construct a word w = 6"ia™i6"2a™2 . . . 
such that A{w) < e and B{w) < e. Since B = 1 — A, this implies that 1 < 2e, a contradiction. 



4.3 Closure under sum 

Lemma 24. PosLimSup and AsLimSup are closed under sum. 

Proof. Given two PosLimSup (resp. AsLimSup) Ai and A2, we construct a PosLimSup (resp. 
AsLimSup) A for the sum of their languages as follows. For a pair {vi , W2) of weights (vi in Ai, for 
1 = 1,2), consider a copy of the synchronized product of Ai and A2. We attach a bit b whose range 
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is {1, 2} to each state to remember that we expect A^, to visit the guessed weight vi,. Whenever this 
occurs, the bit b is set to 3 — 5, and the weight of the state is vi + V2- All other states {i.e. when 
b is unchanged) have weight min{wi + V2 | wi G Vi A V2 £ ^2}- Let the automata constructed 
be Then A = max(„j ^{vi,v2)- Since PosLimSup (resp. AsLimSup) is closed under 

max the result follows. ■ 



Lemma 25. PosLimInf and AsLimInf are closed under sum. 

Proof. Given two PosLlMiNF (resp. AsLimInf) Ai and A2, we construct a PosLlMiNF (resp. 
AsLimInf) a for the sum of their languages as follows. For i = 1, 2, let be the set of weights that 
appear in Ai. Let Umin = minjwi + W2 | e Vi A W2 G V2}. For vi G Vi and V2 G V2, for i = 1, 2, 
consider the PosCW (resp. AsCW) Ay^ obtained from Ai by making all states with weights at least 
Vi as accepting states. Let be the PosCW (resp. AsCW) such that ^(ui.u^) = A^^ n A^^: 

such an PosCW (resp. AsCW) exists since PosCW (resp. AsCW) is closed under intersection. In 
other words, for a word w we have A^^y^ .^,.^■^{w) = 1 iff Ai{w) > vi and ^2(10) > V2. Let A^^^ 
be the PosLiMiNF (resp. AsLimInf) obtained from ^(t,i.i„) by assigning weight vi + V2 to all 
accepting states and weight v„un to all other states. Then the automaton for the sum of Ai and A2 
(denoted as Ai + A2) is max(^,j ^j)^^^ x V2 ^{vi,v2)- Since PosLiMiNF (resp. AsLimInf) is closed 
under max the result follows. ■ 



Lemma 26. AsLimAvg is not closed under sum. 

Proof. Consider the alphabet E = {a, b}, and consider the DLlMAvG-definable languages La and 
Li, that assigns to each word w the long-run average number of a's and 6's in w respectively. Let 

= La + Lb. We show that is not expressible by AsLimAvg. Assume towards contradiction 
that L_|_ is defined by an AsLimAvg A with set of states Q (we assume w.l.o.g that every state in Q 
is reachable). Let /3 > 1 be greater than the maximum absolute value of the weights in A. 

First, we claim that from every state g G Q, if we consider the automaton Aq with q as starting 
state then Aq{a'^) = 1: this follows since if we consider a finite word Wq to reach q, then L+{wq ■ 
a") = 1 and hence A{wq - a") = 1. It follows that from any state q, as k tends to 00, the expected 
average of the weights converges almost-surely to 1 . This implies if we consider the a-Markov chain 
arising from A, then from any state q, for all closed recurrent set C of states reachable from q, the 
expected average of the weights of C is L Hence for every 7 > there exists a natural number fcj 
such that from any state q, for all k > k^ given the word a'' the expected average of the weights is 
at least i with probability 1 — 7 (this is because we can chose long enough k such that the closed 
recurrent states are reached with probability 1 — 7 by property 1 for Markov chains, and then the long 
enough sequence ensures that the expected average approaches 1 by property 2 for Markov chains), 
and for the first fcj steps the expected average of the weights is at least — /3. The same result holds if 
we consider as input a sequence of 6's instead of a's. 

Consider the word w generated inductively by the following procedure: (a) wq is the empty word; 
(b) we generate Wi+i from Wi as follows: {i) the sequence of letters added to Wi to obtain Wi+i is 
at least i; (ii) first we generate a long enough sequence Wi_^_i of a's after Wi such that the average 
number of 6's in Wi ■ w[_^i falls below i; (Hi) then generate a long enough sequence Wi^i of 6's such 
that the average numberof a's in Wi-w[^i '^i+i f^Hs below i; (iv) the word Wi+i = Wi-Wi_^_i -iff+i- 
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The word w is the limit of these sequences. For 7 > 0, consider i > 6 • fcg ■ /? (where fcg satisfies the 
properties described above for 7). By construction for i > 6 • fcj • /3, the length of Wi is at least 6 • fco • /3, 
and hence it follows that in the segment constructed between Wi and Wi+i, for all \wi\ < £ < \wi+i \ 
with probability at least 1 — 7 the expected average of the weights is at least 

e -2 e -23-6' 

Hence for all 7 > 0, the expected average of the weights is at least ^ with probability at least 1 — 7. 
Since this holds for all 7 > 0, it follows that the expected average of the weights is at least ^ almost- 
surely, (i.e., A{w) > ^). We have La{w) = Li,{w) = and thus L+{w) ~ 0, while A{w) > ^. 
Thus we have a contradiction. ■ 



Lemma 27. PosDiSC and AsDiSC are closed under sum. 

Proof. The result for PosDiSC follows from Theorem 5 and the fact that NDiSC and UDiSC are 
closed under sum (which is easy to prove using a synchronized product of automata where the weight 
of a joint transition is the sum of the weights of the corresponding transitions. ■ 

Open question. Whether PosLimAvg is closed under sum remains open. 
5 Decision Problems for Probabilistic Weighted Automata 

We conclude the paper with some decidability and undecidability results for classical decision prob- 
lems about quantitative languages (see Table 1). Most of them are direct corollaries of the results 
in [1]. Given a weighted automaton A and a rational number 1/ G Q, the quantitative emptiness 
problem asks whether there exists a word w G S'^ such that > i^, and the quantitative 

universality problem asks whether La{w) > v for all words w G S'^ . 

Theorem 6. The emptiness and universality problems for PosSUP and AsSuP are decidable. 

Proof. By Theorem 1, these problems reduce to emptiness of DSUP which is decidable ([7, Theo- 
rem 1]). 

The following theorems are trivial corollaries of [1, Theorem 2]. 

Theorem 7. The emptiness problem for PosLimSup and the universality problem for AsLimSup 
are undecidable. 

It is easy to obtain the following result as a straightforward generalization of [1, Theorem 6]. 

Theorem 8. The emptiness problem for AsLimSup and the universality problem for PosLlMiNF 
are decidable. 

Theorem 9. The emptiness problem for PosLlMiNF and the universality problem for AsLimSup 
are decidable. 
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Proof (Sketch). We sketch the main ideas of the proof that emptiness of coBiichi automata in 
positive semantics is achievable in EXPTIME and with exponential memory. The proof extends 
easily to PosLimInf and to the universaHty problem for AsLimSup. 

Emptiness of coBiichi automata in positive semantics can be viewed as deciding the existence 
of a blind positive-winning strategy in a stochastic game with coBiichi objective. It follows from the 
results of [9] that this problem can be decomposed into positive winning for safety and reachability 
objectives. ■ 

The following result is a particular case of [1, Corollary 3]. 

Theorem 10. The emptiness problem for AsLlMiNF and the universality problem for PosLimSup 
are undecidable. 

Finally, by Theorem 5 and the decidability of emptiness for NDiSC, we get the following result. 

Theorem 11. The emptiness problem for PosDiSC and the universality problem for AsDiSC are 
decidable. 

Note that by Theorem 5, the universality problem for NDiSC (which is not know to be decidable) 
can be reduced to the universaUty problem for PosDiSC and to the emptiness problem for AsDiSC. 
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